From Noise to Knowledge: Interactive Summaries for Developer Alerts
Burak Yetiştiren, Hong Jin Kang, Miryung Kim · 2025
Read From Noise to Knowledge: Interactive Summaries for Developer Alerts by Burak Yetiştiren, Hong Jin Kang, Miryung Kim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-07169.txt.
Introduction
Burak Yetiştiren, Hong Jin Kang, and Miryung Kim build a warning browser called SWIRL in a 2025 paper. Bug finding tools flood programmers with warnings to review one by one. Recurring themes among warnings could speed that review. SWIRL groups warnings by rules it learns from the user's own feedback.
As users mark warnings interesting or not, the tool infers shared traits. Groupings highlight structure like containment, subtyping, and called methods. Real warnings from two mature Java projects demonstrated the system. In a user study, programmers explained root causes of boring warnings more confidently with SWIRL. People wanted different groupings, which justified the personalized design. Simulations showed about twelve interactions align the rules with a user's labels. Sensemaking, not just listing, is the product.
Catalog record
| CREATOR | Burak Yetiştiren, Hong Jin Kang, Miryung Kim |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-07169
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Noise to Knowledge: Interactive Summaries for Developer Alerts Burak Yetiştiren, Hong Jin Kang, Miryung Kim Abstract Programmers using bug-finding tools often review their reported warnings one by one. Based on the insight that identifying recurring themes and relationships can enhance the cognitive process of sensemaking , we propose CLARITY , which supports interpreting tool-generated warnings through interactive inquiry. CLARITY derives summary rules for custom grouping of related warnings with active feedback. As users mark warnings as interesting or uninteresting, CLARITY ’s rule inference algorithm surfaces common symptoms, highlighting structural similarities in containment, subtyping, invoked methods, accessed fields, and expressions. We demonstrate CLARITY on Infer and SpotBugs warnings across two mature Java projects. In a within-subject user study with 14 participants, users articulated root causes for similar uninteresting warnings faster and with more confidence using CLARITY . We observed significant individual variation in desired grouping, reinforcing the need for customizable sensemaking. Simulation shows that with rule-level feedback, only 11.8 interactions are needed on average to align all inferred rules with a simulated user’s labels (vs. 17.8 without). Our evaluation suggests that CLARITY ’s active learning-based summarization enhances interactive warning sensemaking. Fig. 1 : CLARITY guides users in sensemaking of tool-generated warnings with inductive summary rules. Users can provide explicit feedback via (A) checkmarking code expressions or (B) highlighting code snippets. CLARITY then refines the summary rules interactively. I Introduction Bug-finding tools, such as Findbugs [ 1 ] and Infer [ 2 ] , have been successful in detecting issues. However, they also produce numerous uninteresting and interesting warnings [ 3 , 4 , 5 , 6 ] . Users often struggle to make sense of them and this difficulty has caused reluctance to adopt these bug-finding tools into a developer workflow [ 3 , 4 , 7 , 3 ] . In our own analysis of the Alibaba Nacos open-source project [ 8 ] , we find that 43% of the 58 Null Pointer Dereference warnings generated by Infer are never fixed, indicating that they are probably ignored by developers. Evidently, the burden of sifting through a large number of tool-generated warnings and the difficulty of sensemaking has impeded the effective adoption of bug finding tools [ 3 , 4 , 5 , 9 ] . Existing research has focused on filtering or reprioritizing warnings [ 10 ] but overlooked the problem of sensemaking, the essential task of making sense of warnings and identifying which warnings should be acted on . Unlike approaches that asks “Is this single warning a false positive or a true bug?”, sensemaking helps developers construct mental models for understanding the code and warnings reported on them. False-positive/false-negative classification is thus a binary, instance-level judgment, whereas sensemaking surfaces higher-level patterns. By elevating the unit of analysis from isolated alerts to configurable groups, CLARITY lets developers reason about whole swaths of findings at once. Inspired by the variational theory of learning [ 11 ] , where a user should form accurate conceptualizations by discerning variations of a phenomenon, we introduce CLARITY ( C lustered L earning for A nalysis and R eview of I nspections for T riage and Y ield). CLARITY surfaces the commonalities and differences between warnings upon interactive feedback from a user. CLARITY empowers developers to create customized groupings. CLARITY ’s interface is shown in Figure 1 . Instead of viewing individual warnings one by one, in CLARITY , a user can formulate customized summaries of warnings by providing interactive feedback. This offers configurability—summary rules are learned from a user’s labeling and code-level feedback. These summary rules highlight common characteristics based on code locations, type hierarchies, and similar expressions. Therefore, the user can easily make sense of similarities and dissimilarities between different groups. CLARITY provides the following three capabilities to improve sensemaking: Feedback-driven Alignment. CLARITY re-infers summary rules, as a user iteratively provides more feedback. This introduces a feedback loop; with more user feedback, the rules become more aligned with the developer intuition of how the warnings are related to each other. Saliency. CLARITY employs inductive logic programming to derive rules over containment, typing relations, and similar code expressions. These relationships capture the implicated code’s scope, type hierarchy, and the API signature of a method where a warning is reported, e.g., the return type of the method, or the class fields that the method uses. This is useful since uninteresting warnings are often correlated by locality and organization of the implicated code [ 12 ] . For example, code located in the same package exposes HTTP endpoints related to unintended remote access [ 13 ] . As another example, a user may wish to group null pointer dereference warnings related to getProperty("os.name") , as it would return a non-null value at runtime [ 14 ] . A user should be able to group warnings based on code cloning relationships [ 15 , 16 , 17 ] or invocation of specific API methods [ 18 ] . Visualization. With this focus on interactive sensemaking, CLARITY ’s user interface is different from a typical user-interface of bug-finding tools such as SpotBugs or Infer. Instead of displaying warnings in a tree view according to predefined filters or organizing them based on file locations, CLARITY organize warnings with custom summaries created with interactive feedback. Fig. 2 : User interface of SpotBugs, a typical bug finding tool, which groups warnings by analysis kind or bug pattern. The lack of configurability for summarizing related warnings forces users to inspect warnings one-by-one. To evaluate CLARITY , we conducted a user study and a simulation experiment. First, we carried out a within-subject user study with 14 participants. The participants inspected null pointer dereference warnings reported by Infer and Spotbugs on two large mature projects. They had to quickly review tool-generated warnings and articulate commonalities between uninteresting warnings. We built a baseline for inspecting warnings one-by-one, similar to conventional tree-based organization bundled with Infer [ 19 ] . A total of 55 different rules were formulated by 14 participants, with an average of 3.9 rules. When using CLARITY , the participants reported lower levels of mental demand (4.8 vs. 5.4 on a 7-point scale) and reported greater confidence (3.4 vs. 2.6 on a 5-point scale), compared to the baseline. Using CLARITY , participants were able to better articulate commonalities among uninteresting warnings 10% more. They also wrote longer, more detailed descriptions about commonalities. Participants also reported that CLARITY ’s customizability is useful for sensemaking warnings. In fact, we found significant individuality among the summary rules constructed by individual participants, indicating that developers would desire greater configurability during their sensemaking process. As a second evaluation method, we conducted simulation experiments to quantify the benefit of rule-based feedback and summarization. When a user provides rule-level feedback as opposed to instance-level feedback, CLARITY ’s resulting rules become aligned faster with interactive feedback. On average, for the warnings found in Alibaba Nacos, within 12 iterations, the rule alignment becomes over 80%, and within 18 iterations, rule alignment becomes 100%. In summary, this paper makes the following contributions: 1. To improve sensemaking of tool-generated warnings, we design an interactive approach that incorporates human feedback and highlights common characteristics among related warnings. Its automated rule-inference groups similar warnings using