TORAI: Multi-source Root Cause Analysis for Blind Spots in Microservice Service Call Graph DOI: 10.1145/3808137 Journal: PACMSE Volume: 3 FSE FSE130 7 fse26mainb-p431-p CCS: Software and its engineering Software reliability CCS: Software and its engineering Software performance
Luan Pham, Huong Ha, Xiuzhen Zhang, Hongyu Zhang · 2026
Read TORAI: Multi-source Root Cause Analysis for Blind Spots in Microservice Service Call Graph DOI: 10.1145/3808137 Journal: PACMSE Volume: 3 FSE FSE130 7 fse26mainb-p431-p CCS: Software and its engineering Software reliability CCS: Software and its engineering Software performance by Luan Pham, Huong Ha, Xiuzhen Zhang, Hongyu Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13522.txt.
Introduction
For microservice systems with untraceable black-box parts, Pham and three colleagues offer TORAI, a tool that hunts the root cause of a failure without a complete call graph. It grades how abnormal each service looks from telemetry, groups services with similar symptoms, and ranks suspects inside each group by causal analysis. Tests on three benchmarks put the true culprit in its top three suggestions.
Catalog record
| CREATOR | Luan Pham, Huong Ha, Xiuzhen Zhang, Hongyu Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 82690 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.13522
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TORAI: Multi-source Root Cause Analysis for Blind Spots in Microservice Service Call Graph DOI: 10.1145/3808137 Journal: PACMSE Volume: 3 FSE FSE130 7 fse26mainb-p431-p CCS: Software and its engineering Software reliability CCS: Software and its engineering Software performance Luan Pham, Huong Ha, Xiuzhen Zhang, Hongyu Zhang Abstract . Existing multi-source root cause analysis (RCA) methods for microservice systems assume all services have traces to construct a service call graph. However, this assumption is not practical as microservice systems evolve rapidly and may contain blackbox services without traces, such as compiled software or unsupported services. We refer to these services as blind spots . In the presence of blind spots, the performance of existing multi-source RCA methods may be affected, as they only diagnose visible services on the call graph. To overcome this limitation, we propose TORAI, a novel unsupervised approach that effectively pinpoints fine-grained root causes without relying on the service call graph. Instead, TORAI first measures anomaly severity using available multi-source telemetry data. It then performs clustering to group services based on their severity symptoms and conducts causal analysis to rank services within each severity cluster. Finally, TORAI aggregates the cluster rankings and uses hypothesis testing to identify fine-grained root causes. TORAI provides an unsupervised approach that leverages available multi-source telemetry data for RCA without requiring a constructed service call graph or further intrusive actions, thus addressing the limitations of existing methods. Our experiments on three benchmark systems demonstrate that TORAI outperforms state-of-the-art baselines remarkably in the presence of blind spots. Performance on real-world failures further shows that TORAI can accurately pinpoint the root causes in top-3 recommendations. † † cc-license: by 1. Introduction Microservices have emerged as a popular form of software systems because of their advantages such as loose coupling, resource flexibility, and simplified deployment processes. However, the inherent complexity of microservice systems often leads to failures, affecting user experience, and causing substantial economic losses. For example, a one-hour downtime on Amazon may cost up to 100 million dollars ( 35 ; 37 ; 40 ) . Therefore, effectively and efficiently diagnosing the root causes of failures in microservice systems is crucial to quickly mitigate the failure and minimize its impact. To effectively conduct root cause analysis (RCA), system operators typically gather three primary sources of telemetry data: metrics, logs, and traces ( 21 ; 62 ; 65 ) . Metrics, typically presented as time series, provide insights into service status, including system-level and application-level metrics like CPU usage and average response time. Logs, comprising semi-structured text, record events that occur at runtime, including request handling events, state changes. Traces, structured in a tree format, record the sequence of user request invocations. Unlike metrics and logs, traces usually require heavy instrumentation with distributed tracing, often involving modifications to microservices’ source code ( 20 ; 46 ; 3 ; 62 ) . Figure 2 A illustrates the multi-source telemetry data. Figure 1. Service Call Graph of the Online Boutique microservice system containing blind spots (i.e., services without distributed tracing instrumentation). We have identified three key limitations of existing multi-source RCA methods ( 21 ; 65 ; 42 ; 59 ; 62 ) : they are either (1) approaches that assume the system has full trace coverage, allowing them to construct a full service call graph from traces, or (2) supervised approaches requiring a large volume of labeled data, or (3) intrusive approaches requiring heavy instrumentation of the microservices’ source code . First, most multi-source RCA methods ( 21 ; 65 ; 42 ; 59 ; 62 ) , except those omitting trace information ( 67 ; 66 ) , assume the system has full trace coverage, i.e., all services in the system are instrumented with distributed tracing, because they rely on traces to construct the service call graph required in their RCA methods. This requirement limits their applicability, especially in large and evolving microservice systems where new services or versions are introduced all the time, the engineers may not have enough effort to implement the distributed tracing for newly introduced services ( 46 ; 3 ) , resulting in many blind spots (i.e., services are not in the service call graph), that directly impact RCA performance, as shown in Figure 1 . Second, many multi-source RCA approaches ( 21 ; 65 ; 27 ; 59 ) are supervised approaches, requiring a large volume of training data to train their models. It is generally impractical to expect such a large volume of training data, as microservice systems are typically large and rapidly developed with many updates ( 46 ; 54 ) . Third, a recent multi-source RCA method ( 62 ) requires a tight integration between traces and logs (i.e., each log line must contain the trace_id ), which demands extensive Q: How do we address the mentioned limitations? We aim to propose a novel multi-source RCA method that satisfies three criteria: (1) avoiding using the call graph constructed from traces; (2) relying on unsupervised techniques; and (3) leveraging available telemetry data without requiring further updates to the source code. effort to implement the necessary changes in the microservices’ source code and may become infeasible when there are closed-source components. Consequently, these three limitations make existing multi-source RCA methods inflexible and challenging to implement in real-world scenarios. In this study, we introduce TORAI, a novel multi-source RCA method for identifying fine-grained root causes of microservice failures. TORAI employs unsupervised techniques intuitively. First, it measures the anomaly severity from multi-source telemetry data. Second, it clusters services with similar severity symptoms. Next, it applies causal analysis to identify coarse-grained root causes (i.e., root cause services). Finally, it uses hypothesis testing to derive fine-grained root causes indicators (e.g., metrics or logs indicating the underlying root cause). TORAI offers several advantages over existing multi-source RCA methods. First, it does not rely on the service call graph (see Figure 1 ), eliminating the need for full trace coverage. Second, it uses unsupervised techniques such as clustering, and causal analysis, removing the need for labeled data and enabling direct application to microservice systems without requiring training. Third, it effectively leverages multi-source telemetry data, allowing for the root cause diagnosis in services not present in the call graph, without the need for additional intrusive instrumentation to achieve full trace coverage. To evaluate TORAI, we conduct extensive experiments on 270 failure cases collected from three benchmark microservice systems and 10 real-world incidents, comparing TORAI against nine state-of-the-art RCA methods. The experimental results demonstrate that TORAI outperforms all baselines in localizing both coarse-grained and fine-grained root causes in terms of effectiveness and efficiency. We further evaluate TORAI on real-world failures from a major internet provider. The results show that TORAI outperforms state-of-the-art baselines and achieves 100% accuracy in ranking the root cause within the top three recommendations. In summary, our main contributions are as follows: • We identify three key limitations of existing multi-source RCA methods, including the assumption of full trace coverage (i.e., assuming the constructed service call graph is complete and contains no blind spots ), the reliance on labeled data, and the requirement for intrusive instrumentation. These limitations motivate our work. • We introduce TORAI, a novel RCA method that addresses the mentioned limitations effectively. TORAI